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[Name of Document] SPECIFICATION 

[Title of the Invention] INFORMATION PROCESSING APPARATUS, 

INFORMATION PROCESSING METHOD, AND RECORDING MEDIUM 
[Claims ] 

[Claim 1] An information processing apparatus for 
receiving the input of a first data stream and a second data 
stream and for connecting the second data stream at a 
predetermined position of the first data stream and 
recording them, the information processing apparatus 
comprising : 

separation means for separating each of the first data 
stream and the second data stream into video data and audio 
data; 

first detection means for detecting the amount of 
deviation, with respect to time, between the video data and 
the audio data of the first data stream, which are separated 
by the separation means, and the amount of deviation, with 
respect to time, between the video data and the audio data 
of the second data stream, which are separated by the 
separation means; 

first computation means for computing the amount of 
shift of the audio data of the second data stream with 
respect to the video data of the second data stream on the 
basis of the two amounts of deviation detected by the first 
detection means; 
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creation means for combining the video data of the 
first data stream and the second data stream, the audio data 
of the first data stream and the second data stream, and 
system data containing the amount of shift computed by the 
first computation means in order to create a third data 
stream; 

control means for controlling, based on the amount of 
shift computed by the first computation means, the creation 
means in such a way that the audio data of the second data 
stream is shifted in time with respect to the video data of 
the second data stream corresponding thereto; and 

recording control means for controlling recording of 
the third data stream created by the creation means onto a 
recording medium. 

[Claim 2] The information processing apparatus according 
to Claim 1, further comprising: 

second detection means for detecting the coding rate of 
the video data of one picture at the connection point of the 
first data stream with respect to the second data stream, 
and the coding rate of the video data of one picture at the 
connection point of the second data stream with respect to 
the first data stream; and 

second computation means for computing, based on the 
two coding rates detected by the second detection means, the 
amount of still image data inserted at the connection point 
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of the first data stream and the second data stream, 

wherein, when it is determined by the second 
computation means that the amount of data is not zero, the 
creation means further combines the still image data, of the 
amount of data computed by the second computation means, 
with the video data of the first data stream and the second 
data stream, the audio data of the first data stream and the 
second data stream, and the system data containing the 
amount of shift computed by the first computation means in 
order to create the third data stream. 

[Claim 3] The information processing apparatus according 
to Claim 1, wherein the audio data of the second data stream 
is shifted by the control means so that the audio data is 
played back continuously without being re-encoded. 

[Claim 4] The information processing apparatus according 
to Claim 1, further comprising: 

reading control means for controlling reading of the 
first data stream recorded on the recording medium; and 

input control means for controlling the input of the 
first data stream read by the reading control means into the 
separation means. 

[Claim 5] The information processing apparatus according 
to Claim 1, wherein the first data stream and the second 
data stream are MPEG data streams. 

[Claim 6] An information processing method for use with 
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an information processing apparatus for receiving the input 
of a first data stream and a second data stream and for 
connecting the second data stream at a predetermined 
position of the first data stream and recording them, the 
information processing method comprising the steps of: 

separating each of the first data stream and the second 
data stream into video data and audio data; 

detecting the amount of deviation, with respect to time, 
between the video data and the audio data of the first data 
stream, which are separated in the separation step, and the 
amount of deviation, with respect to time, between the video 
data and the audio data of the second data stream, which are 
separated in the separation step; 

computing the amount of shift of the audio data of the 
second data stream with respect to the video data of the 
second data stream on the basis of the two amounts of 
deviation detected in the detection step; 

combining the video data of the first data stream and 
the second data stream, the audio data of the first data 
stream and the second data stream, and system data 
containing the amount of shift computed in the computation 
step in order to create a third data stream; 

controlling, based on the amount of shift computed in 
the computation step, the creation step in such a way that 
the audio data of the second data stream is shifted in time 
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with respect to the video data of the second data stream 
corresponding thereto; and 

controlling recording of the third data stream created 
in the creation step on a recording medium. 

[Claim 7] A recording medium having recorded thereon a 
computer- readable program for use with an information 
processing apparatus for receiving the input of a first data 
stream and a second data stream and for connecting the 
second data stream at a predetermined position of the first 
data stream and recording them, the program comprising the 
steps of: 

separating each of the first data stream and the second 
data stream into video data and audio data; 

detecting the amount of deviation, with respect to time, 
between the video data and the audio data of the first data 
stream, which are separated in the separation step, and the 
amount of deviation, with respect to time, between the video 
data and the audio data of the second data stream, which are 
separated in the separation step; 

computing the amount of shift of the audio data of the 
second data stream with respect to the video data of the 
second data stream on the basis of the two amounts of 
deviation detected in the detection step; 

combining the video data of the first data stream and 
the second data stream, the audio data of the first data 
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stream and the second data stream, and system data 
containing the amount of shift computed in the computation 
step in order to create a third data stream; 

controlling, based on the amount of shift computed in 
the computation step, the creation step in such a way that 
the audio data of the second data stream is shifted in time 
with respect to the video data of the second data stream 
corresponding thereto; and 

controlling recording of the third data stream created 
in the creation step onto a recording medium. 

[Claim 8] An information processing apparatus into which 
is loaded a recording medium having recorded thereon a first 
data stream, the information processing apparatus 
comprising: 

recording control means for controlling recording, on 
the recording medium, of a second data stream which is 
connected to a first position of the first data stream so 
that these streams are played back, 

wherein the recording control means controls recording 
of the second data stream in such a way that recording of 
the second data stream is started from a second position 
which is shifted by a predetermined time from the first 
position of the first data stream. 

[Claim 9] An information processing method for use with 
an information processing apparatus into which is loaded a 
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recording medium having recorded thereon a first data stream, 
the information processing method comprising the step of: 

controlling recording, on the recording medium, of a 
second data stream which is connected to a first position of 
the first data stream so that these streams are played back, 

wherein, in the recording control step, recording of 
the second data stream is controlled in such a way that 
recording of the second data stream is started from a second 
position which is shifted by a predetermined time from the 
first position of the first data stream. 

[Claim 10] A recording medium having recorded thereon a 
computer-readable program for use with an information 
processing apparatus into which is loaded a recording medium 
having recorded thereon a first data stream, the program 
comprising the step of: 

controlling recording, on the recording medium, of a 
second data stream which is connected to a first position of 
the first data stream so that these streams are played back, 

wherein, in the recording control step, recording of 
the second data stream is controlled in such a way that 
recording of the second data stream is started from a second 
position which is shifted by a predetermined time from the 
first position of the first data stream. 

[Claim 11] An information processing apparatus into which 
is loaded a recording medium having recorded thereon a first 
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data stream and a second data stream which is connected to a 
predetermined position of the first data stream so that 
these streams are played back, the information processing 
apparatus comprising : 

reading control means for controlling reading of the 
first data stream and the second data stream which are 
recorded on the recording medium; 

separation means for separating each of the first data 
stream and the second data stream, whose reading is 
controlled by the reading control means , to video data and 
audio data; 

first detection means for detecting the amount of 
deviation, with respect to time, between the video data and 
the audio data of the first data stream, which are separated 
by the separation means, and the amount of deviation, with 
respect to time, between the video data and the audio data 
of the second data stream, which are separated by the 
separation means; 

first computation means for computing the amount of 
shift of the audio data of the second data stream with 
respect to the video data of the second data stream on the 
basis of the two amounts of deviation detected by the first 
detection means; 

delaying means for delaying the audio data of the 
second data stream; 
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control means for controlling, based on the amount of 
shift computed by the first computation means, the delaying 
means in such a way that the audio data of the second data 
stream is shifted in time with respect to the video data of 
the second data stream corresponding thereto; and 

rewriting means for rewriting time information 
contained in the video data and the audio data of the second 
data stream. 

[Claim 12] The information processing apparatus according 
to Claim 11, further comprising: 

second detection means for detecting the coding rate of 
the video data of one picture at the connection point of the 
first data stream with respect to the second data stream, 
and the coding rate of the video data of one picture at the 
connection point of the second data stream with respect to 
the first data stream; 

second computation means for computing, based on the 
two coding rates detected by the second detection means, the 
amount of still image data inserted at the connection point 
of the first data stream and the second data stream; and 

insertion means for inserting the still image data of 
the amount of data computed by the second computation means, 
into the connection point between the video data of the 
first data stream and the video data of the second data 
stream when it is determined by the second computation means 
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that the amount of data is not zero. 

[Claim 13] An information processing method for use with 
an information processing apparatus into which is loaded a 
recording medium having recorded thereon a first data stream 
and a second data stream which is connected to a 
predetermined position of the first data stream so that 
these streams are played back, the information processing 
method comprising the steps of : 

controlling reading of the first data stream and the 
second data stream, which are recorded on the recording 
medium; 

separating each of the first data stream and the second 
data stream, whose reading is controlled by the process of 
the reading control step, into video data and audio data; 

detecting the amount of deviation, with respect to time, 
between the video data and the audio data of the first data 
stream, which are separated by the process of the separation 
step, and the amount of deviation, with respect to time, 
between the video data and the audio data of the second data 
stream, which are separated by the process of the separation 
step; 

computing the amount of shift of the audio data of the 
second data stream with respect to the video data of the 
second data stream on the basis of the two amounts of 
deviation detected by the process of the detection step; 
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delaying the audio data of the second data stream; 

controlling, based on the amount of shift computed by 
the process of the computation step, the delaying step in 
such a way that the audio data of the second data stream is 
shifted in time with respect to the video data of the second 
data stream corresponding thereto; and 

rewriting time information contained in the video data 
and the audio data of the second data stream. 

[Claim 14] A recording medium having recorded thereon a 
computer -readable program for use with an information 
processing apparatus into which is loaded a recording medium 
having recorded thereon a first data stream and a second 
data stream which is connected to a predetermined position 
of the first data stream so that these streams are played 
back, the program comprising the steps of: 

controlling reading of the first data stream and the 
second data stream, which are recorded on the recording 
medium; 

separating each of the first data stream and the second 
data stream, whose reading is controlled by the process of 
the reading control step, into video data and audio data; 

detecting the amount of deviation, with respect to time, 
between the video data and the audio data of the first data 
stream, which are separated by the process the separation 
step, and the amount of deviation, with respect to time. 
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between the video data and the audio data of the second data 
stream, which are separated by the process of the separation 
step; 

computing the amount of shift of the audio data of the 
second data stream with respect to the video data of the 
second data stream on the basis of the two amounts of 
deviation detected by the process of the detection step; 

delaying the audio data of the second data stream; 

controlling, based on the amount of shift computed in 
the computation step, the delaying step in such a way that 
the audio data of the second data stream is shifted in time 
with respect to the video data of the second data stream 
corresponding thereto; and 

rewriting time information contained in the video data 
and the audio data of the second data stream. 
[Detailed Description of the Invention] 

[0001] 

[Technical Field of the Invention] 

The present invention relates to an information 
processing apparatus and method, and to a recording medium. 
More particularly, the present invention relates to an 
information processing apparatus and method in which, when 
two MPEG streams are connected, the audio data playback 
timing is shifted so that the audio data can be played back 
continuously so that continuous audio data can be played 
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back at the point where the two MPEG streams are connected, 
and to a recording medium. 
[0002] 

[Description of the Related Art] 

In a case where audio data is compressed, the audio 
data is divided in predetermined units, and compression is 
performed within the divided data. When the audio data 
compressed in this manner is edited and connected, 
connection is possible only in divided data units (that is, 
connection is not possible from the middle of data units). 

[0003] 

The real time length of the data unit in the 
compression of audio data does not match the frame length of 
video data. For example, in MPEG (Moving Picture Coding 
Experts Group/Moving Picture Experts Group), for the video 
data, in a 29.97 Hz system, the number of time stamps (TS) 
of one frame is 3003, whereas for audio data, in an 48 kHz 
MPEG2 Audio Layer II, the number of TSs of one frame is 2160. 

[0004] 

[Problems to be Solved by the Invention] 

For example, in a data stream, such as MPEG, for which 
video and audio are handled at the same time, when data 
streams are connected, since connection is performed by 
using the video data as a reference, at the connection point, 
continuity of the audio data is lost for compressed audio 
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data having a period different from that of the video data. 
[0005] 

Since audio data with lost continuity causes , for 
example, a failure of a decoding buffer to occur, there is a 
possibility that decoding cannot be performed correctly in a 
decoding device . For example , in a case where , in order to 
maintain the continuity of audio data, the phase lag between 
a video signal and an audio signal is admitted and the audio 
signal is directly made continuous, unnatural playback data, 
such as, for example, the audio data being played back prior 
to the video data, might result. 

[0006] 

Furthermore, depending on the video data to be 
connected, there are cases in which, in order to prevent the 
failure of a decoding buffer, a still image whose coding 
rate is close to zero is inserted at a connection point, so 
that the processing of the decoding buffer is in time. In 
such a case, unless audio corresponding to the still image 
to be inserted is provided, a soundless state occurs in the 
connection point, and unnatural playback data results. 

[0007] 

The present invention has been made in view of such 
circumstances. An object of the present invention is to 
play back continuous audio data, without the audio data 
sounding unnatural, at a connection point of two MPEG 
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streams by shifting the audio data playback timing so that 
the audio data can be played back continuously in a case 
where two MPEG streams are connected. 
[0008] 

[Means for Solving the Problems] 

A first information processing apparatus according to 
the present invention includes: separation means for 
separating each of a first data stream and a second data 
stream into video data and audio data; first detection means 
for detecting the amount of deviation, with respect to time, 
between the video data and the audio data of the first data 
stream, which are separated by the separation means, and the 
amount of deviation, with respect to time, between the video 
data and the audio data of the second data stream, which are 
separated by the separation means; first computation means 
for computing the amount of shift of the audio data of the 
second data stream with respect to the video data of the 
second data stream on the basis of the two amounts of 
deviation detected by the first detection means; creation 
means for combining the video data of the first data stream 
and the second data stream, the audio data of the first data 
stream and the second data stream, and system data 
containing the amount of shift computed by the first 
computation means in order to create a third data stream; 
control means for controlling, based on the amount of shift 
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computed by the first computation means, the creation means 
in such a way that the audio data of the second data stream 
is shifted in time with respect to the video data of the 
second data stream corresponding thereto; and recording 
control means for controlling recording of the third data 
stream created by the creation means onto a recording medium. 
[0009] 

The information processing apparatus may further 
include second detection means for detecting the coding rate 
of the video data of one picture at the connection point of 
the first data stream with respect to the second data stream, 
and the coding rate of the video data of one picture at the 
connection point of the second data stream with respect to 
the first data stream; and second computation means for 
computing, based on the two coding rates detected by the 
second detection means, the amount of still image data 
inserted at the connection point of the first data stream 
and the second data stream, wherein, when it is determined 
by the second computation means that the amount of data is 
not zero, the creation means may further combine the still 
image data of the amount of data computed by the second 
computation means, with the video data of the first data 
stream and the second data stream, the audio data of the 
first data stream and the second data stream, and the system 
data containing the amount of shift computed by the first 
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computation means in order to create a third data stream. 
[0010] 

The audio data of the second data stream may be shifted 
by the control means in such a way that the audio data is 
played back continuously without being re-encoded from the 
second data stream. 

[0011] 

The information processing apparatus may further 
comprise reading control means for controlling reading of 
the first data stream recorded on the recording medium; and 
input control means for controlling the input of the first 
data stream read by the reading control means into the 
separation means. 

[0012] 

The first data stream and the second data stream may be 
MPEG data streams. 
[0013] 

A first image processing method according to the 
present invention includes the steps of: separating each of 
a first data stream and a second data stream into video data 
and audio data; detecting the amount of deviation, with 
respect to time, between the video data and the audio data 
of the first data stream, which are separated in the 
separation step, and the amount of deviation, with respect 
to time, between the video data and the audio data of the 
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second data stream, which are separated in the separation 
step; computing the amount of shift of the audio data of the 
second data stream with respect to the video data of the 
second data stream on the basis of the two amounts of 
deviation detected in the detection step; combining the 
video data of the first data stream and the second data 
stream, the audio data of the first data stream and the 
second data stream, and system data containing the amount of 
shift computed in the computation step in order to create a 
third data stream; controlling, based on the amount of shift 
computed in the computation step, the creation step in such 
a way that the audio data of the second data stream is 
shifted in time with respect to the video data of the second 
data stream corresponding thereto; and controlling recording 
of the third data stream created in the creation step on a 
recording medium. 
[0014] 

A program recorded on a first recording medium includes 
the steps of: separating each of a first data stream and a 
second data stream into video data and audio data; detecting 
the amount of deviation, with respect to time, between the 
video data and the audio data of the first data stream, 
which are separated in the separation step, and the amount 
of deviation, with respect to time, between the video data 
and the audio data of the second data stream, which are 
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separated in the separation step; computing the amount of 
shift of the audio data of the second data stream with 
respect to the video data of the second data stream on the 
basis of the two amounts of deviation detected in the 
detection step; combining the video data of the first data 
stream and the second data stream, the audio data of the 
first data stream and the second data stream, and system 
data containing the amount of shift computed in the 
computation step in order to create a third data stream; 
controlling, based on the amount of shift computed in the 
computation step, the creation step in such a way that the 
audio data of the second data stream is shifted in time with 
respect to the video data of the second data stream 
corresponding thereto; and controlling recording of the 
third data stream created in the creation step on a 
recording medium. 
[0015] 

A second information processing apparatus according to 
the present invention includes recording control means for 
controlling recording, on a recording medium, of a second 
data stream which is connected to a first position of a 
first data stream so that these streams are played back, 
wherein the recording control means controls recording of 
the second data stream in such a way that recording of the 
second data stream is started from a second position which 
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is shifted by a predetermined time from the first position 
of the first data stream. 
[0016] 

A second information processing method according to the 
present invention includes the step of controlling recording, 
on a recording medium, of a second data stream which is 
connected to a first position of a first data stream so that 
these streams are played back, wherein, in the recording 
control step, recording of the second data stream is 
controlled in such a way that recording of the second data 
stream is started from a second position which is shifted by 
a predetermined time from the first position of the first 
data stream. 

[0017] 

A program recorded on a second recording medium 
includes the step of: controlling recording, on a recording 
medium, of a second data stream which is connected to a 
first position of a first data stream so that these streams 
are played back, wherein, in the recording control step, 
recording of the second data stream is controlled in such a 
way that recording of the second data stream is started from 
a second position which is shifted by a predetermined time 
from the first position of the first data stream. 

[0018] 

A third information processing apparatus according to 
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the present invention includes: reading control means for 
controlling reading of a first data stream and a second data 
stream which are recorded on a recording medium; separation 
means for separating each of the first data stream and the 
second data stream, whose reading is controlled by the 
reading control means, to video data and audio data; first 
detection means for detecting the amount of deviation, with 
respect to time, between the video data and the audio data 
of the first data stream, which are separated by the 
separation means, and the amount of deviation, with respect 
to time, between the video data and the audio data of the 
second data stream, which are separated by the separation 
means; first computation means for computing the amount of 
shift of the audio data of the second data stream with 
respect to the video data of the second data stream on the 
basis of the two amounts of deviation detected by the first 
detection means; delaying means for delaying the audio data 
of the second data stream; control means for controlling, 
based on the amount of shift computed by the first 
computation means, the delaying means in such a way that the 
audio data of the second data stream is shifted in time with 
respect to the video data of the second data stream 
corresponding thereto; and rewriting means for rewriting 
time information contained in the video data and the audio 
data of the second data stream. 
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[0019] 

The information processing apparatus may further 
include second detection means for detecting the coding rate 
of the video data of one picture at the connection point of 
the first data stream with respect to the second data stream, 
and the coding rate of the video data of one picture at the 
connection point of the second data stream with respect to 
the first data stream; second computation means for 
computing, based on the two coding rates detected by the 
second detection means, the amount of still image data 
inserted at the connection point of the first data stream 
and the second data stream; and insertion means for 
inserting the still image data of the amount of data 
computed by the second computation means, into the 
connection point between the video data of the first data 
stream and the video data of the second data stream when it 
is determined by the second computation means that the 
amount of data is not zero. 

[0020] 

A third information processing method according to the 
present invention includes the steps of : controlling reading 
of a first data stream and a second data stream, which are 
recorded on a recording medium; separating each of the first 
data stream and the second data stream, whose reading is 
controlled by the process of the reading control step, into 
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video data and audio data; detecting the amount of deviation, 
with respect to time, between the video data and the audio 
data of the first data stream, which are separated by the 
process of the separation step, and the amount of deviation, 
with respect to time, between the video data and the audio 
data of the second data stream, which are separated by the 
process of the separation step; computing the amount of 
shift of the audio data of the second data stream with 
respect to the video data of the second data stream on the 
basis of the two amounts of deviation detected by the 
process of the detection step; delaying the audio data of 
the second data stream; controlling, based on the amount of 
shift computed by the process of the computation step, the 
delaying step in such a way that the audio data of the 
second data stream is shifted in time with respect to the 
video data of the second data stream corresponding thereto; 
and rewriting time information contained in the video data 
and the audio data of the second data stream. 
[0021] 

A third program recorded on a third recording medium 
having recorded thereon a program, the program includes the 
steps of: controlling reading of a first data stream and a 
second data stream, which are recorded on a recording 
medium; separating each of the first data stream and the 
second data stream, whose reading is controlled by the 
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process of the reading control step, into video data and 
audio data; detecting the amount of deviation, with respect 
to time, between the video data and the audio data of the 
first data stream, which are separated by the process of the 
separation step, and the amount of deviation, with respect 
to time, between the video data and the audio data of the 
second data stream, which are separated by the process of 
the separation step; computing the amount of shift of the 
audio data of the second data stream with respect to the 
video data of the second data stream on the basis of the two 
amounts of deviation detected by the process of the 
detection step; delaying the audio data of the second data 
stream; controlling, based on the amount of shift computed 
in the computation step, the delaying step in such a way 
that the audio data of the second data stream is shifted in 
time with respect to the video data of the second data 
stream corresponding thereto; and rewriting time information 
contained in the video data and the audio data of the second 
data stream. 
[0022] 

In the first information processing apparatus, the 
first image processing method, and the program recorded on 
the first recording medium according to the present 
invention, each of a first data stream and a second data 
stream is separated into video data and audio data. The 
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amount of deviation with respect to time between the 
separated video data and audio data of the first data stream 
and the amount of deviation with respect to time between the 
separated video data and audio data of the second data 
stream are detected. Based on the two detected amounts of 
deviation, the amount of shift of the audio data of the 
second data stream with respect to the video data of the 
second data stream is detected. The video data of the first 
data stream and the second data stream, the audio data of 
the first data stream and the second data stream, and system 
data containing the computed amount of shift are combined to 
create a third data stream. Based on the computed amount of 
shift, the audio data of the second data stream is 
controlled to be shifted in time with respect to the 
corresponding video data of the second data stream, and 
recording of the third created data stream on a recording 
medium is controlled. 
[0023] 

In the second information processing apparatus, the 
second image processing method, and the program recorded on 
the second recording medium according to the present 
invention, the recording, onto a recording medium, of the 
second data stream which is connected at the first position 
of the first data stream so that these streams are played 
back is controlled. The recording of the second data stream 
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is controlled so that the recording of the second data 
stream is started from a second position deviated by a 
predetermined time from the first position of the first data 
stream. 

[0024] 

In the third information processing apparatus, the 
third image processing method, and the program recorded on 
the third recording medium according to the present 
invention, the reading of the first data stream and the 
second data stream which are recorded on the recording 
medium is controlled. Each of the read first data stream 
and the read second data stream is separated into video data 
and audio data. The amount of deviation with respect to 
time between the separated video data and audio data of the 
first data stream, and the amount of deviation with respect 
to time between the separated video data and audio data of 
the second data stream are detected. Based on the two 
amounts of deviation, the amount of shift of the audio data 
of the second data stream with respect to the video data of 
the second data stream is computed. The audio data of the 
second data stream is delayed. Based on the computed amount 
of shift , delay is controlled so that the audio data of the 
second data stream is shifted in time with respect to the 
video data of the second data stream corresponding thereto. 
The time information contained in the video data and the 
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audio data of the second data stream is rewritten. 
[0025] 

[Description of the Embodiments] 

The embodiments of the present invention will now be 
described below with reference to the drawings. 
[0026] 

Fig. 1 is a block diagram showing the construction of a 
recording data creation section 1 for recording input MPEG 
data on a magnetic tape 2 in a magnetic tape recording 
apparatus or a magnetic tape playback apparatus according to 
the present invention. 

[0027] 

An MPEG stream input to the recording data creation 
section 1 is input to a demultiplexer 11, and the 
demultiplexer 11 demultiplexes the input MPEG stream into a 
PES (Packetized Elementary Stream) video signal and a PES 
audio signal. The PES video signal demultiplexed by the 
demultiplexer 11 is input to a video buffer 12, and the PES 
audio signal demultiplexed by the demultiplexer 11 is input 
to an audio buffer 13. The video buffer 12 and the audio 
buffer 13 store the input signals and output them in 
accordance with the TS contained in the PES header of each 
signal. 

[0028] 

From the data header of the signal stored in the video 
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buffer 12, a detection section 14 detects, a PTS 
(Presentation Time Stamp), a DTS (Decoding Time Stamp), an 
SCR (System Clock Reference) (or a PCR (Program Clock 
Reference)), a VBV (Video Buffer Verifier) delay value, and 
a bit rate; detects the PTS, the bit rate, the VBV delay 
value, and the amount of offset from the data header of the 
signal stored in the audio buffer 13; and outputs them to a 
control section 15 . 
[0029] 

When the delay section 16 receives the input of the 
video signal output by the video buffer 12, the delay 
section 16 delays the video signal by a predetermined time. 
The control section 15 detects the amount of deviation 
between the video data and the audio data based on the data 
input from the detection section 14, creates AUX (auxiliary) 
data (containing the data of the detected amount of 
deviation) (described later by using Fig. 3), to be added to 
the data created in a multiplexer 17, and outputs it to the 
multiplexer 17. The control section 15, where necessary, 
generates a control signal for controlling the addition of a 
copy picture (described later by using Fig. 5), an audio 
signal corresponding to the lag between the video signal and 
the audio signal, or a control signal for controlling the 
addition of an audio signal corresponding to the copy 
picture, and outputs it to the multiplexer 17. 
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[0030] 

The multiplexer 17 combines, in accordance with the 
control signal input from the control section 15, the video 
signal input from the delay section 16, the audio signal 
input from the audio buffer 13, and system data (containing 
AUX data) input from the control section 15. When a control 
signal indicating the addition of a copy picture is input 
from a control section 44, the multiplexer 17 adds, in 
accordance with this signal, a copy picture (described later 
by using Fig. 5). When a control signal indicating the 
addition of audio data is input , audio data corresponding to 
a copy picture is added in accordance with this signal and 
is output to an ECC (Error Check and Correct) section 18. 
The ECC section 18 adds, to a data stream input from the 
multiplexer 17, an ECC (Error Correcting Code), which is a 
redundancy code which is added separately from the main data, 
in order to correct errors, such as bit garbling, and 
outputs it to a data conversion section 19. 

[0031] 

The data conversion section 19 performs, on the input 
data, 24-25 conversion such that a single redundant bit 
selected so that, for example, the pilot components for 
tracking appear strongly or performs randomizing such that 1 
or 0 does not continue, modulates the data according to a 
method suitable for recording on the magnetic tape 2 (for 
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example, NRZ (Non-Return to Zero)), amplifies it, and 
outputs it to a magnetic head 20. The magnetic head 20 
records the data on the magnetic tape 2 in accordance with 
the signal input from the data conversion section 19. 
[0032] 

Furthermore, a drive 3 is connected to the control 
section 15, and a magnetic disk 21, an optical disk 22, a 
magneto -optical disk 23, or a semiconductor memory 24 is 
loaded into the drive 3 where necessary, so that data is 
exchanged . 

[0033] 

Fig. 2 is a block diagram showing the construction of a 
recording data playback section 31 for playing back and 
outputting data recorded on the magnetic tape 2 or for 
combining by a multiplexer 32 and outputting it as an MPEG 
stream in a magnetic tape playback apparatus or a magnetic 
tape recording/playback apparatus. Components in Fig. 2 
corresponding to those of Fig. 1 are given the same 
reference numerals, and accordingly, descriptions thereof 
are omitted where appropriate. 

[0034] 

The data recorded on the magnetic tape 2 is read by the 
magnetic head 20 and is output to <a data conversion section 
41. The data conversion section 41 performs 25-24 
conversion such that, for example, a single redundant bit. 
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which is selected and is added so that pilot components for 
tracking appear strongly, is deleted from the input data, 
derandomizes the data recorded on the magnetic tape 2 
because the data recorded on the magnetic tape 2 has been 
subjected to randomizing such that conversion is performed 
so that 1 or 0 does not continue, performs format conversion 
thereon, outputs the data to an ECC and data separation 
section 42. 
[0035] 

The ECC and data separation section 42 performs error 
checking and correction by using an ECC added to the input 
data, separates it into a video signal and an audio signal, 
outputs the video signal to a video buffer 45, and outputs 
the audio signal to an audio buffer 48, 

[0036] 

A detection section 43 detects the PTS, the DTS, the 
SCR, the VBV delay value, and the bit rate from the AUX 
packet of the video signal within the data of the ECC and 
data separation section 42, detects the PTS, the bit rate, 
the VBV delay value, and the amount of offset from the AUX 
packet of the audio signal, and outputs them to a control 
section 44. 

[0037] 

The control section 44 detects the amount of deviation 
between the video data and the audio data on the basis of 
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the data input from the detection section 43, creates a PES 
header to be added to the data created in a delay and data 
addition section 46 and in a delay and data addition section 
49, outputs it to the delay and data addition section 46 and 
the delay and data addition section 49, creates, where 
necessary, a control signal for controlling the addition of 
a copy picture (described later by using Fig. 5), an audio 
signal corresponding to the deviation between the video 
signal and the audio signal, or a control signal for 
controlling the addition of an audio signal corresponding to 
a copy picture, and outputs it to the delay and data 
addition section 46 and the delay and data addition section 
49. 

[0038] 

The video buffer 45 and the audio buffer 48 store the 
input signal, and, in accordance with the TS contained in 
the AUX data, output it to the delay and data addition 
section 46 and the delay and data addition section 49. When 
the delay and data addition section 46 receives the input of 
the video signal output by the video buffer 45, the delay 
and data addition section 46 delays the video signal by a 
predetermined time in accordance with the control signal 
input from the control section 44. When a control signal 
indicating the addition of a copy picture is input from the 
control section 44, the delay and data addition section 46 
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adds, in accordance with this signal, a copy picture 
(described later by using Fig. 5), and outputs it to a TS 
correction section 47. Also, when the delay and data 
addition section 49 receives the input of the audio signal 
output by the audio buffer 48, the delay and data addition 
section 49 delays, in accordance with the control signal 
input from the control section 44, the audio signal by a 
predetermined time, and when a control signal indicating the 
addition of audio data is input from the control section 44, 
the delay and data addition section 49 adds, in accordance 
with this signal, the audio data corresponding to the copy 
picture, and outputs it to a TS correction section 50. 
[0039] 

The TS correction section 47 and the TS correction 
section 50 refer to the TS of the data supplied from the 
delay and data addition section 46 and the delay and data 
addition section 49 in order to correct the value of the TS 
which is deviated due to the processing of the delay and 
data addition section 46 and the delay and data addition 
section 49, and output them as a PES video signal and a PES 
audio signal to a multiplexer 32 or a decoder 33. 

[0040] 

The multiplexer 32 combines the input PES video signal 
and the input PES audio signal in order to create an MPEG 
stream, and outputs it. The decoder 33 decodes the input 
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PES video signal and the input PES audio signal, outputs the 
decoded video signal to, for example, a monitor (not shown) 
so as to display it, and outputs the decoded audio signal to, 
for example, a speaker (not shown) so as to produce audio 
output . 

[0041] 

Although in Figs. 1 and 2, the construction for 
recording an MPEG stream on the magnetic tape 2 and the 
construction for playing back the data recorded on the 
magnetic tape 2 are described separately, the present 
invention can be applied to any one of a magnetic tape 
recording apparatus having only the recording data creation 
section 1 described using Fig. 1, a magnetic tape playback 
apparatus having only the recording data playback section 31 
described using Fig. 2, and a magnetic tape 
recording/playback apparatus (or a magnetic tape editing 
apparatus) having both the recording data creation section 1 
and the recording data playback section 31 described using 
Figs. 1 and 2. 

[0042] 

Next, referring to Fig. 3, data recorded on the 
magnetic tape 2 will be described. 
[0043] 

An MPEG stream is separated into audio AUX data, audio 
data, video AUX data, and video data, having a predetermined 
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amount of data, by the recording data creation section 1 
described using Fig. 1, is converted into a format in which 
recording is possible on the magnetic tape 2, and is 
repeatedly recorded in a time series on the magnetic tape 2 . 
The video AUX data contains the values of the PTS, the DTS, 
the SCR, the bit rate, the VBV delay, and the offset of the 
video data which follows immediately after the video AUX 
data. The audio AUX data contains the values of the PTS, 
the bit rate, the VBV delay, and the offset of the audio 
data which follows immediately after the audio AUX data. 
[0044] 

Then, the data recorded on the magnetic tape 2 is read 
in a time series in the direction of travel of the tape in 
the playback data creation section described using Fig. 2, 
video data is played back based on the values of the PTS, 
the DTS, the SCR, the bit rate, the VBV delay, and the 
offset, contained in the video AUX data, and audio data is 
played back based on the values of the PTS, the bit rate, 
the VBV delay, and the offset, contained in the audio AUX 
data. 

[0045] 

Next, referring to Figs. 4 to 7, a description will be 
given of a case in which different streams are tag-recorded 
(edited) . 

[0046] 



- 35 - 



For example, in a case where it is desired to connect a 
stream 2 such as that shown in Fig. 4(B) to a stream 1 such 
as that shown in Fig. 4(A) and to edit these streams, 
editing is performed based on the video data. That is, 
editing is performed in such a way that a video 3 of the 
stream 1 of Fig. 4(A) becomes continuous with the video I of 
the stream 2 of Fig. 4(B). Here, in a portion where the 
extending-of f portion of audio 5 with respect to the video 3 
of the stream 1 overlaps the audio I of the stream 2 , if the 
audio I is erased, the audio data loses continuity, and 
unnatural playback data in which audio is interrupted 
results . 

[0047] 

For this reason, like a stream 3 shown in Fig. 4(C), by 
shifting the audio I to the audio III of the stream 2 
backward in the direction of a time axis so that the audio I 
of the stream 2 is connected to the audio 5 of the stream 1, 
playback data which is not unnatural can be obtained. 

[0048] 

Also, in order to prevent an overflow of a video 
decoder buffer (for example, the video buffer 45 described 
using Fig. 2) for buffering video data during playback, a 
method is sometimes used in which an overflow is prevented 
in such a way that a still image having a very small coding 
rate (for example, the same image data as the image data 
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immediately before the connection point of the video data 
(hereinafter referred to as a "copy picture")) is inserted 
to a connection point so that the processing termination of 
the video decoder buffer is awaited. While the inserted 
copy picture is being played back, for example, if a 
soundless state occurs or the audio data before the 
connection point is repeatedly played back, unnatural audio 
data results. 
[0049] 

In a point immediately after the video 3 (picture B) (a 
point indicated by x in the figure) of the stream 1 shown in 
Fig. 5(A), when a stream 2 shown in Fig. 5(B) is connected 
and edited, a copy picture is inserted between the video 3 
and the video I (picture I). For this reason, like the 
stream 3 shown in Fig. 5(C), by using audio 6, which is 
audio data, corresponding to the video 4 (picture I or 
picture P) of the stream 1 to be deleted, as audio data of 
the connection point of the stream 3 without deleting the 
audio 6, playback data which is not unnatural can be 
obtained. The insertion of the audio 6 between the audio 5 
and the audio I is, to be specific, to shift the playback 
position of the audio I to after the audio 6. 

[0050] 

Also, in a point immediately after the video 2 (a point 
indicated by x in the figure) of the stream 1 shown in Fig. 
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6(A), when the portion of and subsequent to the video II of 
the stream 2 shown in Fig. 6(B) (the point indicated by y in 
the figure) is connected and edited, and the stream 3 shown 
in Fig. 6(C) is created, if a no- sound portion occurs like 
the audio stream 1 of Fig. 6(C), unnatural playback data 
results. Furthermore, like the audio stream 2, a method of 
making audio III directly continuous with the stream 3 may 
be used. The proceeding of the audio data earlier than the 
video data is not possible in a natural state, and unnatural 
playback data still results. For this reason, as shown in 
the audio stream 3, by using the audio 4, which is the audio 
data at the connection point of the video 3 to be deleted, 
of the stream 1 without deleting the audio 4 (by inserting 
the audio 4 between the audio 3 and the audio III), playback 
data which is not unnatural can be obtained. The insertion 
of the audio 4 between the audio 3 and the audio III is, to 
be specific, to shift the playback position of the audio III 
to after the audio 4. 
[0051] 

The audio data used for the connection point, described 
using Figs. 5 and 6, basically corresponds to the video data 
to be deleted by editing. That is, in the data, recorded on 
the magnetic tape 2, shown in Fig. 7, in a case where it is 
desired to connect video data to video n which is at the 
recording termination point of the main stream 1, and to 
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erase the video data of video n+1 and subsequent video data, 
there occurs a necessity of causing several pieces of audio 
data within the audio data of audio n+1 and subsequent audio 
data to remain as audio data for copy pictures without being 
erased (over-recording) . 
[0052] 

The editing process described using Figs. 4 to 7 may be 
performed during data recording time or during data playback 
time. 

[0053] 

More specifically, in a case where the stream 2 is 
connected after predetermined video data of the stream 1 as 
base data and is edited, as shown in Fig. 8, in the 
recording data creation section 1 described using Fig. 1, 
the above -described copy picture and the audio data for the 
copy picture, corresponding to the copy picture, may be used 
so that the video data and the audio data are rearranged 
like connected data arrangement images and is recorded on 
the magnetic tape 2. During playback, in the recording data 
playback section 31 described using Fig. 2, the video data 
and the audio data may be output as is recorded on the 
magnetic tape 2 . 

[0054] 

In a case where each of the stream 1 and the stream 2 
is input, as an MPEG input, to the recording data creation 
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section 1, under the control of the control section 15, the 
addition of a copy picture at the connection point and the 
rearrangement of data are performed in the multiplexer 17, 
and the rearranged data is recorded on the magnetic tape 2 
via the ECC section 18, the data conversion section 19, and 
the magnetic head 20. 
[0055] 

Furthermore, in a case where the stream 1 is recorded 
on the magnetic tape 2 in advance, for example, in the 
recording data playback section 31 of the magnetic tape 
recording/playback apparatus (magnetic tape editing 
apparatus) having both the recording data creation section 1 
and the recording data playback section 31, the data 
recorded on the magnetic tape 2 may be read once and 
converted into an MPEG stream and then may be input to the 
recording data creation section 1 by a multiplexer 32. Then, 
under the control of the control section 15, the addition of 
the copy picture and the rearrangement of the data may be 
performed in the multiplexer 17. 

[0056] 

In this manner, in a case where the data is rearranged 
like connected data arrangement images and is recorded, 
since the data becomes continuous on the magnetic tape 2, 
the arrangement of the data does not vary, irrespective of 
where playback is started. Furthermore, the synchronization 
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with another data stream can be easily realized. 
[0057] 

Alternatively, as shown in Fig. 9, in the recording 
data creation section 1 described using Fig. 1, during 
recording onto the magnetic tape 2, a predetermined spacing 
may be provided from the final data of the stream 1 in order 
to secure audio for a copy picture, and the stream 2 may be 
overwritten. In the recording data playback section 31 
described using Fig. 2, during playback, the video data and 
the audio data may be rearranged and played back like 
connected data arrangement images . 

[0058] 

During playback, when the data is rearranged like 
connected data arrangement images, a stream recording 
process can be simplified in comparison with a case in which 
the data is rearranged like connected data arrangement 
images and is recorded. Also, since the data recorded on 
the magnetic tape 2 is very close to the data before being 
edited, even when a data playback method or a data editing 
method is changed in the future, it is easy to deal with the 
change . 

[0059] 

In this case, the amount of over-recording described 
using Fig. 7 becomes a problem. That is, if attempts are 
made to cause a lot of remaining audio data to remain, a lot 



- 41 - 



of invalid video data also remains , and the recording 
capacity of the magnetic tape 2 is wasted. As shown in Fig. 
10 , when it is desired to connect the stream 2 to 
immediately after the video 2 recorded on the magnetic tape 
2 and to edit it (that is, the position indicated by point A 
in the figure becomes the connection point), the amounts of 
the remaining audio data and the invalid video data differ 
depending on the position where overwriting of the stream 2 
is started. 
[0060] 

Fig. 11 shows the amounts of remaining audio data and 
invalid video data in a case where the overwriting of the 
stream 2 is started at points A to F in Fig. 10. As shown 
in Fig. 11, when the overwrite starting point is point A 
(that is, when the stream 2 is overwritten immediately after 
the connection point), invalid video data does not occur, 
and there is also no remaining audio data, thereby making 
playback of the audio data at the connection point difficult. 
When the overwrite starting point is set after the 
connection point A with respect to time, invalid video data 
does occur, and the remaining audio data remains on the 
magnetic tape 2, thereby allowing the audio data to be 
played back like connected data arrangement images during 
playback. The number of remaining pieces of remaining audio 
data may be set to a predetermined number (for example, 9 in 
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the number of TSs), and may be set by a user. 
[0061] 

Next, referring to Fig. 12, a description will be given 
of a process for creating connected data arrangement images, 
performed by the detection section 14 and the control 
section 15 of the recording data creation section 1 
described using Fig. 1, or performed by the detection 
section 43 and the control section 44 of the recording data 
playback section 31 described using Fig. 2. 

[0062] 

In step SI, the detection section 14 of the recording 
data creation section 1 (or the detection section 43 of the 
recording data playback section 31, and the same applies 
hereinafter) detects the amount of VBV delay, the PTS, and 
the bit rate of an I picture or a P picture (that is, a 
picture which is erased after being connected) , which is the 
next picture after the connection point of the stream 1, and 
outputs them to the control section 15 of the recording data 
creation section 1 (or the control section 44 of the 
recording data playback section 31, and the same applies 
hereinafter). In step S2, the detection section 14 detects 
the PTS of the start audio frame corresponding to the 
picture for which the amount of VBV delay, the PTS, and the 
bit rate have been detected in step SI, detects the amount 
of deviation and the amount of offset from the value of the 
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PTS, and outputs them to the control section 15. As shown 
in Fig. 13, the amount of deviation of the stream 1 is the 
amount of deviation between the final video frame at the 
connection point and the audio frame corresponding thereto. 
[0063] 

The detection section 14 detects the amount of VBV 
delay, the PTS, and the bit rate of the start I picture of 
the stream 2 in step S3, detects the PTS of the audio frame 
corresponding to the start I picture of the stream 2, and 
detects the amount of deviation and the amount of offset 
from the value of the PTS in step S4. As shown in Fig. 13, 
the amount of deviation of the stream 2 is the amount of 
deviation of the start position of the audio data 
corresponding to the start video data which is connected at 
the connection point, an amount Ou of offset being contained 
in this amount of deviation. 

[0064] 

In step S5, the control section 15 determines the 
number Ncp of copy pictures and the amount Sf of stuffing, 
on the basis of the amount of VBV delay and the bit rate of 
the I picture or the P picture of the stream 1 , and on the 
basis of the amount of VBV delay and the bit rate of the 
start I picture of the stream 2. 

[0065] 

More specifically, if the coding rate of the picture of 
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the stream 1, which is erased after tag-recording, is 
denoted as Fs, the coding rate Fs is given by equation (1) 
below: 

Fs = ds x Rs (1) 
where ds is assumed to be the amount of VBV delay of 
the stream 1, detected in step SI, and Rs is assumed to be 
the bit rate of the stream 1, detected in step SI. 
[0066] 

Then, if the coding rate of the start I picture of the 
stream 2, which is overwritten during tag-editing, is 
denoted as Fu, the coding rate Fu is given by equation (2) 
below: 

Fu = du x Ru (2) 
where du is assumed to be the amount of VBV delay of 
the stream 2, detected in step S3, and Ru is assumed to be 
the bit rate of the stream 2, detected in step S3. 
[0067] 

Then, under the condition of Fu > Fs, the minimum 
number Ncp of copy pictures and the amount Sf of stuffing, 
which satisfy the following equation (3), are determined: 

Fu = Fs + Ncp x 1 picture time x Rs - Sf (3) 

[0068] 

Furthermore, under the condition of Fu =s Fs , the number 
Ncp of copy pictures and the amount Sf of stuffing satisfy 
the following equations (4): 
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Fu = Fs - Sf, Ncp =0 (4) 
[0069] 

In step S6 , the control section 15 tentatively 
determines the number Nca of pieces of audio data to be 
provided on the basis of the remaining audio data of the 
stream 1 recorded by over- recording described using Fig. 8. 
More specifically, if the number of TSs of one frame of the 
video data is 3003 and the number of TSs of one frame of the 
audio data is 2160, a minimum number Nca of pieces of audio 
data, which satisfies the following equation (5), is 
computed as shown below: 

Psv + Ncp x 3003 <s Psa + Nca x 2160 (5) 
where Psv indicates the PTS value of the I picture or the P 
picture to be erased, detected in step SI, and Psa indicates 
the PTS value of the audio data corresponding to the I 
picture or the P picture to be erased, detected in step S2. 

[0070] 

In step S7, the control section 15 computes the phases 
of the copy picture and the audio data, that is, the amount 
H of extending-of f shown in Fig. 13, on the basis of the 
amount of deviation and the amount of offset of the audio 
frame corresponding to the final picture of the stream 1, 
the amount of deviation and the amount of offset of the 
audio frame corresponding to the start I picture of the 
stream 2, and the number of pieces of audio data, which has 
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been tentatively determined in step S6 . The amount H of 
extending-of f is computed based on the following equation 
(6): 

H = (Psa + Nca x 2160) - (Psv + Ncp x 3003) (6) 
[0071] 

In step S8 , the control section 15 computes a new audio 
frame accumulated offset amount Or shown in Fig. 13 based on 
the detection result in step S3, the detection result in 
step S4 , and the detection result in step S7 . The new audio 
frame accumulated offset amount Or is expressed based on the 
following equation (7): 

Or = Ou + H - (Puv - Pua) (7) 
where Ou indicates the value of the offset of the stream 2, 
detected in step S4 , Puv indicates the PTS value of the 
start I picture of the stream 2, detected in step S3, and 
Pua indicates the PTS value of the audio data of the start I 
picture of the stream 2, detected in step S4. 

[0072] 

In step S9 , the control section 15 subtracts the new 
audio frame accumulated offset amount Or from one audio 
frame length, and determines in step S10 whether or not the 
subtracted result is smaller than 0 . 

[0073] 

When it is determined in step S10 that the subtracted 
result is smaller than 0, since the position of the start 
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audio frame of the stream 2 leads the original position, the 
control section 15 sets Nca = Nca + 1 in step S12, the 
process returns to step S7, and processing of step S7 and 
subsequent steps is repeated* 
[0074] 

When it is determined in step S10 that the subtracted 
result is not smaller than 0, the control section 15 
determines in step S12 whether or not the subtracted result 
is greater than one audio frame. 

[0075] 

When it is determined in step SI 2 that the subtracted 
result is greater than one audio frame, since extra audio 
data for a copy picture has been added, the control section 
15 sets Nca = Nca - 1 in step S13, the process returns to 
step S7, and processing of step S7 and subsequent steps is 
repeated. 

[0076] 

When it is determined in step S12 that the subtracted 
result is not greater than one audio frame, since the number 
Nca of pieces of audio data for copy pictures is the optimal 
number, the control section 15 determines the number Nca of 
pieces of audio data in step S14, and the processing is 
terminated. 

[0077] 

According to the processing which has thus been 
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described, even if, as described using Fig. 8, data is 
rearranged like connected data arrangement images during 
recording of data on the magnetic tape 2, and even if, as 
described using Fig. 9, data is rearranged like connected 
data arrangement images when data is played back from a 
magnetic tape, the connected audio data can be played back 
continuously. Thus, problems, such as audio data being 
connected unnaturally or audio data at the connection point 
and subsequent thereto being played back earlier than the 
corresponding video data, do not occur. In addition, since 
audio data is connected together without performing re- 
encoding, degradation in sound quality does not occur. 
[0078] 

Furthermore, since the phase lag between video data and 
audio data, which occurs due to editing, is stored by being 
contained in data and is thereby managed, it is possible to 
return to the original after editing and to limit the amount 
of deviation, where necessary. The management of the phase 
lag between the video data and the audio data is not always 
necessary for all the apparatuses receiving these data 
streams . 

[0079] 

The above-described series of processes can also be 
performed by software. For the software, programs which 
form the software are installed from a recording medium into 
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a computer incorporated into dedicated hardware or into, for 
example, a general-purpose personal computer capable of 
executing various types of functions by installing various 
programs . 

[0080] 

This recording medium, as shown in Fig. 1 and 2, is 
constructed by packaged media formed of the magnetic disk 21 
(including a floppy disk), the optical disk 22 (including a 
CD-ROM (Compact Disk-Read Only Memory), and a DVD (Digital 
Versatile Disc)), the magneto-optical disk 23 (including an 
MD (Mini-Disk)), or the semiconductor memory 24, in which 
programs are recorded, the packaged media being distributed 
separately from the computer so as to provide programs to a 
user. 

[0081] 

In this specification, steps describing a program 
recorded in a recording medium contain not only processing 
performed on a time -series manner along the described 
sequence, but also processing performed in parallel or 
individually, although the processing is not necessarily 
performed in a time -series manner. 
[0082] 
[Advantages ] 

According to the first information processing apparatus, 
the first image processing method, and the program recorded 
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on the first recording medium in accordance with the present 
invention, each of a first data stream and a second data 
stream is separated into video data and audio data. The 
amount of deviation with respect to time between the 
separated video data and audio data of the first data stream 
and the amount of deviation with respect to time between the 
separated video data and audio data of the second data 
stream are detected. Based on the two detected amounts of 
deviation, the amount of shift of the audio data of the 
second data stream with respect to the video data of the 
second data stream is detected. The video data of the first 
data stream and the second data stream, the audio data of 
the first data stream and the second data stream, and system 
data containing the computed amount of shift are combined to 
create a third data stream. Based on the computed amount of 
shift, the audio data of the second data stream is 
controlled to be shifted in time with respect to the 
corresponding video data of the second data stream, and 
recording of the third created data stream on a recording 
medium is controlled. Therefore, natural audio data can be 
played back at the connection point of the two MPEG streams 
without causing a failure in the buffer, and the amount of 
shift can be managed also in the third created stream. 
[0083] 

According to the second information processing 



- 51 - 



apparatus, the second image processing method, and the 
program recorded on the second recording medium in 
accordance with the present invention, the recording, onto a 
recording medium, of the second data stream which is 
connected at the first position of the first data stream so 
that these streams are played back is controlled. The 
recording of the second data stream is controlled so that 
the recording of the second data stream is started from a 
second position deviated by a predetermined time from the 
first position of the first data stream. Therefore, when 
the recorded data is to be played back, since audio data 
corresponding to the image data to be erased of the first 
stream can be played back at the connection point of the 
first stream and the second stream, it is possible to record 
recording data from which natural audio data can be 
continuously played back. 
[0084] 

According to the third information processing apparatus, 
the third image processing method, and the program recorded 
on the third recording medium in accordance with the present 
invention, the reading of the first data stream and the 
second data stream which are recorded on the recording 
medium is controlled. Each of the read first data stream 
and the read second data stream is separated into video data 
and audio data. The amount of deviation with respect to 
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time between the separated video data and audio data of the 
first data stream, and the amount of deviation with respect 
to time between the separated video data and audio data of 
the second data stream are detected. Based on the two 
amounts of deviation, the amount of shift of the audio data 
of the second data stream with respect to the video data of 
the second data stream is computed. The audio data of the 
second data stream is delayed. Based on the computed amount 
of shift, delay is controlled so that the audio data of the 
second data stream is shifted in time with respect to the 
video data of the second data stream corresponding thereto. 
The time information contained in the video data and the 
audio data of the second data stream is rewritten. 
Therefore, since audio data corresponding to image data to 
be erased of the first stream can be played back at the 
connection point of the first stream and the second stream, 
it is possible to continuously play back natural playback 
data. 

[Brief Description of the Drawings] 
[Fig. 1] 

Fig. 1 is a block diagram showing the construction of a 
recording data creation section. 
[Fig. 2] 

Fig. 2 is a block diagram showing the construction of a 
recording data playback section. 
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[Fig. 3] 

Fig. 3 is a diagram illustrating data recorded on a 
magnetic tape . 
[Fig. 4] 

Fig. 4 is a diagram illustrating the shift of audio 
data when two MPEG streams are connected. 
[Fig. 5] 

Fig. 5 is a diagram illustrating a copy picture and 
addition of audio data corresponding thereto when two MPEG 
streams are connected. 
[Fig. 6] 

Fig. 6 is a diagram illustrating the shift of audio 
data when two MPEG streams are connected. 
[Fig. 7] 

Fig. 7 is a diagram illustrating over -recording. 
[Fig. 8] 

Fig. 8 is a diagram illustrating a case in which input 
data is rearranged like connected data arrangement images 
and is recorded. 
[Fig. 9] 

Fig. 9 is a diagram illustrating a case in which the 
recorded data is rearranged like connected data arrangement 
images and is played back. 
[Fig. 10] 

Fig. 10 is a diagram illustrating an overwrite starting 
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point in the case of Fig. 9, 
[Fig. 11] 

Fig. 11 is a diagram illustrating remaining audio data 
and invalid video data, corresponding to the overwrite 
starting point of Fig. 10. 
[Fig. 12] 

Fig. 12 is a flowchart illustrating a process for 
arranging data like connected data arrangement images . 
[Fig. 13] 

Fig. 13 is a diagram illustrating the amount of 
deviation, the offset, the number of copy pictures, and the 
number of pieces of secured audio data of the data arranged 
like connected data arrangement images . 
[Reference Numerals] 

1 recording data creation section, 2 magnetic tape, 
11 demultiplexer, 12 video buffer, 13 audio buffer, 14 
detection section, 15 control section, 16 delay section, 
17 multiplexer, 31 recording data playback section, 42 
ECC and data separation section, 43 detection section, 44 
control section, 45 video buffer, 46 delay and data 
addition section, 47 TS correction section, 48 audio 
buffer, 49 delay and data addition section, 50 TS 
correction section 
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[Name of Document] ABSTRACT 
[Abstract ] 

[Object] When MPEG streams are connected, audio data is 
connected without losing continuity, so that playback data 
which is not unnatural is obtained. 

[Solving Means] When data subsequent to video II of a 
stream 2 of Fig. 6(B) is connected to immediately after 
video 2 of a stream 1 of Fig. 6(A) and is edited to create a 
stream 3 of Fig. 6(C), unnatural playback data results when 
a no- sound portion occurs as in an audio stream 1 of Fig. 
6(C) and when audio III is directly connected to a stream 3 
and the audio data leads the video data as in an audio 
stream 2. Therefore, by inserting audio 4 corresponding to 
video 3 to be erased of the stream 1 , as audio data at the 
connection point of the stream 3, between the audio 3 and 
the audio III, playback data which is not unnatural is 
obtained. 

[Selected Figure] Fig. 6 
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